Video conferencing technology can be used to provide audio and video features to communication parties. For example, business associates can use a video conferencing system to host strategic meetings. Video conferencing systems can be used to enable interactions between two people in distant rooms (point-to-point) or more than one person in a number of rooms at several sites (multi-point). Encoding and decoding techniques are used to improve a user experience when using a video conferencing system. For example, codecs can be designed to encode and decode a captured data stream or signal. Additionally, rate distortion optimizations can be used to control bit allocation operations as part of distributing a given bandwidth budget. For example, rate distortion optimizations can be used to distribute a bandwidth budget across a picture sequence, group of pictures (GOP), picture frames or fields, picture slices (e.g., portions of a picture), and/or macro blocks (MB) (e.g., a 16×16 sets of pixels).